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DETAILED ACTION 

1. This office action is in response to application No. 10/813055 filed 
3/31/2004. Claims 1-28 are pending and have been examined. 

Priority 

2. Receipt is acknowledged of papers submitted under 35 U.S.C. 119(a)- 
(d), which papers have been placed of record in the file. 

Drawings 

3. Figures 8 and 29 should be designated by a legend such as —Prior Art- 
- because only that which is old is illustrated. See MPEP § 608.02(g). 
Corrected drawings in compliance with 37 CFR 1.121(d) are required in reply 
to the Office action to avoid abandonment of the application. The 
replacement sheet(s) should be labeled "Replacement Sheet" in the page 
header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the 
drawing figures. If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in 
the next Office action. The objection to the drawings will not be held in 
abeyance. 
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Allowable Subject Matter 

4. Claims 11-13, 19, 21 and 23 objected to as being dependent upon a 
rejected base claim, but would be allowable if rewritten in independent form 
including all of the limitations of the base claim and any intervening claims. 



Claim Rejections - 35 USC §102 

5. The following is a quotation of the appropriate paragraphs of 35 

U.S.C. 102 that form the basis for the rejections under this section made in 

this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country 
or in public use or on sale in this country, more than one year prior to the date of application 
for patent in the United States. 

6. Claims 1-4 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Katagiri (5,966,112). 

Apropos claim 1, Katagiri teaches: 
A display device (Title), comprising: 

an array substrate having display elements (LCD 102, Fig 2) and 
output units configured to output binary image pickup data; 

an image processing unit (108, Fig. 2) configured to have a 
bidirectional bus for a CPU (bidirectional/ connected to 105, Fig. 2); and 
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an LCDC (106,107 and 109, Fig. 2)which has a bidirectional bus for 
said CPU (bidirectionaly connected to 105, Fig. 2). 

Apropos claim 2, Katagiri teaches: 

The display device according to claim 1, wherein said LCDC has a receiving 
unit (107 holds display information, col 8 lines 59-60) configured to receive 
the output of an image pickup unit provided separate from said display 
device (receives external image signals Col 8 line 59-61). 

Apropos claim 3, Katagiri teaches: 
A display device, comprising: 

an array substrate having display elements (LCD 102, Fig 2) and 
output units configured to output binary image pickup data; 

an image processing unit (108, Fig. 2) configured to have a 
bidirectional bus for a CPU (bidirectionaly connected to 105, Fig. 2). 

Apropos claim 4, Katagiri teaches: 

The display device according to claim 3, wherein said LCDC has a receiving 
unit (107 holds display information, col 8 lines 59-60) configured to receive 
the output of an image pickup unit provided separate from said display 
device (receives external image signals Col 8 line 59-61). 
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7. Claim 5,20,22,24,26 rejected under 35 U.S.C. 102(b) as being 
anticipated by Ogawa (6,243,069). 

Apropos claim 5, Ogawa teaches: 
A display device, comprising: 

display devices in pixels (pixel 14, Fig. 2) formed in vicinity of 
intersections of signal lines (22, Fig. 2) and scanning lines (23, Fig. 2) 
disposed in length and breadth; 

image pickup units (photodiode 25, Fig 2), at least one of said image 
pickup units being provided corresponding to each pixel (See Fig. 2 one 
photodiode 25 per pixel 14), and each conducting image pickup at a 
prescribed range; 

binary data storages (Line memory 36, Fig. 2) which store binary data 
corresponding to results of image picked up by said image pickup unit (Col 
12 lines 23-25); and 

an array substrate which outputs the binary data in multiple pixels that 
do not neighbor to each other in at least one direction of length or breadth 
direction (See Fig. 6 and Col 17 line 39-44). 

Apropos claim 20, Ogawa teaches: 
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A display device, comprising: 

display devices in pixels (pixel 14, Fig. 2) formed in vicinity of 
intersections of signal lines (22, Fig. 2) and scanning lines (23, Fig. 2) 
disposed in length and breadth; 

image pickup units (photodiode 25, Fig 2), at least one of said image 
pickup units being provided corresponding to each pixel (See Fig. 2 one 
photodiode 25 per pixel 14), and each conducting image pickup at a 
prescribed range; 

binary data storages (Line memory 36, Fig. 2) which store binary data 
corresponding to results of image picked up by said image pickup unit (Col 
12 lines 23-25); and 

a multiple gradation data generator which generates multiple 
gradation data with first, second third colors based on the binary data with 
the first, second and third colors picked up by said image pickup unit (Image 
data is obtained for each color; Abstract lines 12-15); and 
a color composition unit configured to generate image pickup data with a 
fourth color based on the multiple gradation data with the first, second and 
third colors (Since Ogawa captures RGB data, it's inherint that the image 
formed will obtain white data). 



Apropos claim 22, Ogawa teaches: 
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The display device according to claim 20, wherein said color composition unit 
generates the image pickup data with red, green and blue colors 
components based on the multiple gradation data with the first, second and 
third colors (Abstract lines 12-15). 

Apropos claim 24, Ogawa teaches: 

The display device according to claim 20, further comprising a backlight (18 
Fig. 1) device capable of alternately illuminating the lights with the first, 
second and third colors (Abstract lines 8-10), said backlight device being 
disposed on back face of an insulation substrate on which said display 
elements and said image pickup units are provided (See Fig. 1, backlight 18 
disposed behind glass 12 of display), wherein said image pickup unit 
repeatedly conducts image pickup with respect to the first, second and third 
colors of said backlight device (Abstract lines 12-15). 

Apropos claim 26, Ogawa teaches: 

The display device according to claim 20, wherein each pixel is substantially 
square shape (Pixels shown in Fig. 3 are substantially square). 
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Claim Rejections - 35 USC §103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis 

for all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a 
whole would have been obvious at the time the invention was made to a person having 
ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 

9. The factual inquiries set forth in Graham v. John Deere Co., 
383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a 
background for determining obviousness under 35 U.S.C. 103(a) are 
summarized as follows: 

1. Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims 
at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application 
indicating obviousness or nonobviousness. 



10. Claims 6 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ogawa in view of Yanof (2004/0085458) and Makinouchi (2004/0204060). 



Apropos claim 6, Ogawa teaches: 
A display device, comprising: 

a pixel array unit having display elements (pixel 14, Fig. 2) formed in 
vicinity of intersections of signal lines (22, Fig. 2) and scanning lines (23, 
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Fig. 2) disposed in length and breadth, image pickup units (photodiode 25, 
Fig 2) and an output unit which outputs binary data corresponding to image 
picked up by said image pickup unit (A/D converter 36, Fig. 2); 
However Ogawa fails to explicitly teach: 

a first image processing unit configured to generate multiple gradation 
data based on multiple binary data picked up by said image pickup units 
based on multiple image pickup conditions ; and 

a second image processing unit configured to receive either the image 
pickup data picked up by said image pickup device or the multiple gradation 
data generated by said first image processing unit, to conduct a prescribed 
image processing. 

In the same field of image capturing Yanof teaches a digital imaging 
system that first processes gradation data in an Automatic exposure control 
circuit (132 Fig. 1 [0025]). Yanof also teaches other processes that receive 
the image picked up by the digital imaging device as performs further 
processing such as color interpolation or gamma correction (140 and 144, 
Fig. 1). 

Therefore it would have been obvious to one of ordinary skill in the art 
at the time of the invention to supply image processing units to generate 
gradation data and gamma correct etc as taught by Yanof in the image 
capture device of Ogawa in order to capture higher quality images ([0013]). 
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Ogawa and Yanof both fail to explicitly teach: 

a image pickup device provided separate from said image pickup unit; 

In the same field of devices with liquid crystal displays, Makinouchi 
teaches a cellular phone that has a camera and a liquid crystal display 
[0019]. 

Therefore it would have been obvious at the time of the invention to 
use the liquid crystal display as taught by Ogawa in the cellular phone 
taught by Makinouchi in order to provide a communications device that can 
take facial images and still provide high density document scanning. 

Apropos claim 7, Yanof further teaches: 

The display device according to claim 6, wherein said second image 
processing unit conducts at least one of gradation correction, color 
correction (color interpolation 140, Fig. 1), defective pixel correction, edge 
correction and noise correction. 

Apropos claim 8, Ogawa and Yanof teach: 

The display device according to claim 6, wherein said pixel array unit is 
formed on an insulation substrate using TFTs (Thin Film Transistors) (glass 
12, Fig. 4 with TFTs 27, Fig. 4 and 5); and 
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said first image processing unit is a semiconductor chip (processing 
units on silicon chip [0024]). 

Apropos claim 9, Ogawa teaches: 

The display device according to claim 8, further comprising a display 
controller IC which embeds said first image processing unit and supplies 
digital pixel data for said pixel array unit to said pixel array unit (Control 71, 
Fig. 2 and Col 10 line 65-68). 

Apropos claim 10, Ogawa further teaches: 

The display device according to claim 6, wherein said image pickup unit 
arranged to a direction that the signal lines align is disposed in zigzag form 
for each pixel (Col 19 lines 42-45). 

11. Claims 14-16 and 27 rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ogawa (6,243,069) in view of Choi (6,791,520). 

Apropos claim 14, Ogawa teaches: 
A display device, comprising: 
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display devices in pixels (pixel 14, Fig. 2) formed in vicinity of 
intersections of signal lines (22, Fig. 2) and scanning lines (23, Fig. 2) 
disposed in length and breadth; 

image pickup units (photodiode 25, Fig 2), at least one of said image 
pickup units being provided corresponding to each pixel (See Fig. 2 one 
photodiode 25 per pixel 14), and each conducting image pickup at a 
prescribed range; 

binary data storages (Line memory 36, Fig. 2) which store binary data 
corresponding to results of image picked up by said image pickup unit (Col 
12 lines 23-25); and 

However Ogawa fails to explicitly teach: 

an averaging gradation estimation unit configured to estimate an 
averaging gradation of whole display screen based on the binary data of the 
pixels connected to a portion of the scanning lines which do not neighbor to 
each other. 

In the same field of Liquid crystal displays, Choi teaches a method of 
measuring designated points across a display surface and calculating the 
average luminance thereof (Col 6 lines 13-15). 

Therefore it would have been obvious to one of ordinary skill in the art 
at the time of the invention to have an average luminance estimation unit as 
taught by Choi in the liquid crystal display device of Ogawa in order to detect 
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image sticking defects and luminance change ratios of the display (Col 5 line 
54-55). 

Apropos claim 15, Choi further teaches: 

The display device according to claim 14, wherein said averaging gradation 
estimation unit estimates the averaging gradation of the whole display 
screen based on the results of the image picked up by said image pickup 
unit corresponding to a portion of the signal lines which do not neighbor to 
each other (Choi measures a plurality of points to determine luminance as 
shown in Fig. 7 these points do not neighbor one another). 

Apropos claim 16, Choi further teaches: 

The display device according to claim 14, wherein said averaging gradation 
estimation unit estimates the averaging gradation of the whole display 
screen based on the binary data of the pixels connected to the scanning 
lines disposed for every m pieces (m is an integer not less than 2) and the 
signal lines disposed for every n pieces (n is an integer not less than 2). 
(luminance detect areas shown in Fig. 7 appear to be arranged such that 
every mth scan line and nth signal line is detected) 

Apropos claim 27, Ogawa fails to explicitly teach: 
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The display device according to claim 20, further comprising an averaging 
gradation estimation unit configured to estimate the averaging gradation of 
the whole display screen based on the binary data of the pixel data 
connected to a portion of the scanning lines which do not neighbor to each 
other and a portion of the signal lines which do not neighbor to each other. 

In the same field of Liquid crystal displays, Choi teaches a method of 
measuring designated points across a display surface and calculating the 
average luminance thereof (Col 6 lines 13-15). 

Therefore it would have been obvious to one of ordinary skill in the art 
at the time of the invention to have an average luminance estimation unit as 
taught by Choi in the liquid crystal display device of Ogawa in order to detect 
image sticking defects and luminance change ratios of the display (Col 5 line 
54-55). 

12. Claim 17 and 18 rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ogawa and Choi as applied to claim 14 above, and further 
in view of Yanof. 

Apropos claim 17, Ogawa fails to explicitly teach: 

The display device according to claim 14, further comprising; 
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a signal processing output circuit which converts the binary data for 
multiple pixels into serial data and outputs the serial data; and 

an output determination unit configured to determine whether or not 
to output the image pickup data of the remaining image pickup unit from 
said signal processing output circuit, based on the estimation result of said 
averaging gradation estimation unit. 

In the same field of image capture devices, Yanof teaches correcting 
the exposure control of a image capture device in order to obtain a better 
picture, the picture received is not further processed till the exposure is 
within acceptable levels (See Fig. 2, process loops till no more corrections of 
exposure are required). Yanof also teaches that storage media are for image 
capture devises are in serial format (such as through a USB, [0028]) 

Therefore it would have been obvious to one of ordinary skill in the art 
at the time of the invention to have a deterimation unit to only allow the 
final picture to be rendered if the exposure was deemed acceptialbe as 
taught by Yanof in the image capture device of Ogawa in order to improve 
image quality. Also, it would have been obvious to provide a processing 
circuit that converts the image data into serial data in order to allow the 
images captured to be stored or shared via USB link, etc. 

« 

Apropos claim 18, Yanof further teaches: 
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The display device according to claim 17, wherein said image pickup unit 
newly conducts image pickup while changing image pickup conditions when 
determined that said output determination unit does not output the image 
pickup data of said image pickup unit (exposure values adjusted till image is 
at desired level, Fig. 2). 



13. Claim 25 and 28 rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ogawa in view of Yanof. 

Apropos claim 25, Ogawa teaches: 

The display device according to claim 20, 

said image pickup unit repeatedly picks up image with respect to the 
cases where illumination color of said backlight are the first, second and 
third colors (Abstract lines 8-10). 

However Ogawa fails to explicitly teach: 

wherein said image pickup unit repeatedly picks up the image on 
multiple image pickup conditions with respect to the first, second and third 
colors of said backlight device; 
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In the same field of image capture devices, Yanof teaches a method of 
repeatedly capturing an image while varying exposure control till exposure is 
within an acceptable level (Fig. 2). 

Therefore it would have been obvious to one of ordinary skill in the art 
at the time of the invention to perform repeated image captures for 
exposure control as taught by Yanof, on each of the color images captured 
by Ogawa in order to improve image quality. 

Apropos claim 28, Ogawa fails to explicitly teach: 

The display device according to claim 20, further comprising; 

a signal processing output circuit which converts the binary data for 
multiple pixels into serial data and outputs the serial data; and 

an output determination unit configured to determine whether or not 
to output the image pickup data of the remaining image pickup unit from 
said signal processing output circuit, based on the estimation result of said 
averaging gradation estimation unit. 

In the same field of image capture devices, Yanof teaches correcting 
the exposure control of a image capture device in order to obtain a better 
picture, the picture received is not further processed till the exposure is 
within acceptable levels (See Fig. 2, process loops till no more corrections of 



Application/Control Number: 10/813,055 Page 18 

Art Unit: 2629 

exposure are required). Yanof also teaches that storage media are for image 
capture devises are in serial format (such as through a USB, [0028]) 

Therefore it would have been obvious to one of ordinary skill in 
the art at the time of the invention to have a determination unit to only 
allow the final picture to be rendered if the exposure was deemed acceptable 
as taught by Yanof in the image capture device of Ogawa in order to 
improve image quality. Also, it would have been obvious to provide a 
processing circuit that converts the image data into serial data in order to 
allow the images captured to be stored or shared via USB link, etc. 

Conclusion 

Any inquiry concerning this communication or earlier communications 
from the examiner should be directed to Randal L. Willis whose telephone 
number is (571) 270-1461. The examiner can normally be reached on 
Monday to Friday from 7:30am to 5:00pm (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Amr Awad can be reached on (571) 272-7764. The 
fax phone number for the organization where this application or proceeding 
is assigned is 571-273-8300. 



Application/Control Number: 10/813,055 



Page 19 



Art Unit: 2629 

Information regarding the status of an application may be obtained 
from the Patent Application Information Retrieval (PAIR) system. Status 
information for published applications may be obtained from either Private 
PAIR or Public PAIR. Status information for unpublished applications is 
available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on 
access to the Private PAIR system, contact the Electronic Business Center 
(EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272- 
1000. 
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